1. Introduction
This case study applies that lifecycle to a healthcare problem: estimating whether an adult patient discharged from a tertiary-care hospital in Karachi is likely to be readmitted within 30 days of the index admission. The patient-level data are synthetic; the healthcare setting, terminology, and selected population-level figures are grounded in published Pakistani evidence.
By the end of this case, you should be able to look at a healthcare ML problem and ask not only “Which algorithm should we use?” but also “What exactly are we predicting, what information is valid at discharge, how should success be measured, which patients might be affected by errors, and how will we know the system remains useful?”
2. The Problem: Who Is Likely to Return to the Hospital?
Tertiary hospitals in Karachi manage large numbers of patients with chronic and acute illnesses. A patient may be discharged after an admission for pneumonia, heart failure, uncontrolled hypertension, diabetes-related complications, kidney disease, or another medical condition. Some patients recover without further hospital care. Others return within a short period of time.
A hospital wants to identify patients who may benefit from additional discharge planning, medication review, follow-up, or other support. The objective is not to replace clinical judgment. Instead, the model is intended to help the care team decide where limited post-discharge resources may be most useful.
2.1 A Real-World Karachi Context
A 2026 retrospective study conducted at a tertiary-care hospital in Karachi examined 35,496 adult internal-medicine admissions from 2016 to 2020. It reported that 2,822 patients, or 8.0%, were readmitted within 30 days of the index admission. The mean age of the overall study population was 55.3 years.
The dataset used throughout this case study is synthetic and does not contain real patient records. The values are designed to represent a realistic hospital setting while protecting patient privacy.
2.2 Why These Patients Are Not Just “Rows in a Dataset”
Healthcare decisions affect people directly. A prediction may trigger a follow-up call, a medication review, a primary-care appointment, additional laboratory testing, or closer monitoring. This means the project has several stakeholders with different concerns.
- Doctors and nurses who need clinically useful information without unnecessary alerts.
- Case managers who may coordinate post-discharge support.
- Hospital administrators who care about bed utilization, readmissions, and resource allocation.
- Patients and families whose care and follow-up may be affected by the prediction.
- Data scientists and ML engineers who build, evaluate, deploy, and monitor the system.
2.3 The Business and Clinical Objective
The hospital does not simply want a model with a high classification score. It wants a useful way to identify patients who may need additional attention after discharge.
Project question: Can information available during the hospital stay and at discharge be used to estimate the likelihood that an adult patient will be readmitted within 30 days, so that the hospital can prioritize appropriate post-discharge support?
At the time of prediction, only information available during the hospital stay and at discharge can be used. Information generated after discharge cannot be used as a predictor.
3. Defining the ML Task
3.1 What Is One Prediction?
One prediction corresponds to one index hospital admission. The prediction is generated close to discharge and estimates whether the same patient will have an unplanned readmission to the hospital within the next 30 days.
This immediately creates a boundary between two periods:
- Before prediction: patient history, admission details, diagnoses, laboratory results, length of stay, and other information available by discharge.
- After prediction: what happens to the patient during the following 30 days.
The second period defines the target. It should not quietly leak back into the features.
3.2 Classification or Regression?
The primary task is binary classification:
- 1: readmitted within 30 days.
- 0: not readmitted within 30 days.
We could formulate other questions—such as predicting the number of days until readmission or the expected number of future admissions—but the first version is intentionally simple, keeping the ML lifecycle at the center of the problem.
3.3 What Does “Readmission” Mean?
A target variable must have an operational definition. For this case, define readmission as an unplanned hospital admission within 30 days of discharge from the index admission. Planned admissions for scheduled procedures would be excluded.
This definition is not a minor administrative detail. Changing it changes the labels, the prevalence of the target, and potentially the behavior the model learns.
4. Understanding the Data
4.1 A Synthetic Karachi Hospital Dataset
Imagine that a Karachi tertiary-care hospital has assembled a historical dataset of adult medical admissions. Each row represents one index admission and contains information available by discharge.
Illustrative patient-level data:
| admission_id | age | sex | diabetes | hypertension | ischemic_heart_disease | ckd | prior_admissions_12m | length_of_stay | emergency_admission | discharge_to_home | readmitted_30d |
|---|---|---|---|---|---|---|---|---|---|---|---|
| KR10231 | 67 | F | 1 | 1 | 0 | 1 | 2 | 7 | 1 | 1 | 1 |
| KR10232 | 44 | M | 0 | 1 | 0 | 0 | 0 | 3 | 0 | 1 | 0 |
| KR10233 | 58 | M | 1 | 1 | 1 | 0 | 1 | 9 | 1 | 1 | 1 |
| KR10234 | 31 | F | 0 | 0 | 0 | 0 | 0 | 2 | 0 | 1 | 0 |
| KR10235 | 72 | M | 1 | 1 | 1 | 1 | 3 | 11 | 1 | 1 | 1 |
Diabetes, hypertension, ischemic heart disease, and kidney disease are highly relevant conditions in Pakistani clinical settings. Published work from a tertiary-care center in Karachi has documented diabetes mellitus, dyslipidemia, and ischemic heart disease among patients presenting with uncontrolled hypertension. These conditions provide clinically meaningful variables for this case study.
4.2 A More Complete Data Dictionary
| Variable | Meaning | Type |
|---|---|---|
| age | Patient age at index admission | Numeric |
| sex | Recorded sex | Categorical |
| diabetes | Documented diabetes mellitus | Binary |
| hypertension | Documented hypertension | Binary |
| ischemic_heart_disease | Documented ischemic heart disease | Binary |
| ckd | Documented chronic kidney disease | Binary |
| prior_admissions_12m | Number of hospital admissions during the previous 12 months | Numeric |
| length_of_stay | Number of days in the index admission | Numeric |
| emergency_admission | Whether the index admission originated through emergency services | Binary |
| discharge_to_home | Whether the patient was discharged home rather than to another care setting | Binary |
| readmitted_30d | Unplanned hospital readmission within 30 days | Target |
4.3 Features That May Need More Thought
Some variables are useful but require context. For example, length of stay may contain valuable information about illness severity, but it is only legitimate if the prediction is made at discharge. If the hospital wants to make the prediction 24 hours after admission, then using the final length of stay would be leakage because that value is not yet known.
4.4 A Potential Leakage Variable
Consider this variable:
| Variable | Why it looks useful | Should it be used? |
|---|---|---|
| readmission_flag_from_followup_call | It directly indicates whether the patient returned to hospital. | No. It is derived from the future outcome. |
| post_discharge_emergency_visit | It is highly related to future readmission. | No. It occurs after the prediction point. |
| discharge_day_length_of_stay | It summarizes the completed index stay. | Yes, when prediction is made at discharge. |
The same rule applies as in the finance case:
Predictive does not mean permissible. A variable can be extremely predictive of the target and still be invalid if it was not available when the prediction had to be made.
5. Exploratory Data Analysis
5.1 How Common Is Readmission?
Suppose the dataset contains 10,000 historical index admissions and 800 are followed by an unplanned readmission within 30 days. The observed readmission rate would therefore be:
Readmission rate = 800 / 10,000 = 8%
This is close to the 8.0% 30-day readmission rate reported in the 2026 Karachi tertiary-hospital study cited earlier. Our 10,000-record teaching dataset is nevertheless synthetic and should not be interpreted as evidence about any particular hospital.
5.2 What Should We Examine?
Before fitting a model, we might compare readmitted and non-readmitted patients by:
- age and age group;
- number of prior admissions;
- length of stay;
- diabetes, hypertension, ischemic heart disease, and chronic kidney disease;
- emergency versus elective admission;
- discharge destination;
- missingness in laboratory or administrative variables.
5.3 Why Missingness Deserves Attention
Healthcare data often contain missing measurements because a test was not ordered, a measurement was unavailable, or a clinical workflow differed from patient to patient. Missingness may therefore be related to the underlying clinical process rather than being pure random noise.
For example, suppose serum creatinine is missing more often for patients who were treated in settings where the test was not considered necessary. Blindly replacing every missing value with the overall mean may hide useful information about the data-generation process.
6. Designing the Data Pipeline
6.1 Train, Validation, and Test Data
Imagine that the 10,000 historical admissions are split into training, validation, and test sets. The test set should remain untouched until the model-development decisions are complete.
In healthcare, the split strategy can become particularly important when patient records span time. If the hospital's clinical practice, patient mix, or documentation system changes over the years, a purely random split may make the problem easier than a true future deployment scenario.
6.2 A More Realistic Thought Experiment
Suppose we train on admissions from 2016–2019 and test on admissions from 2020. Now the test set approximates a future population rather than a random sample from the same historical period.
This leads to an important question:
If our model performs well when tested on patients from the same period as the training data, does that prove it will work equally well when hospital policies, patient populations, treatment patterns, or documentation systems change?
7. Establishing a Baseline
Suppose 8% of admissions in the training data are followed by readmission. A trivial baseline that predicts “No Readmission” for everyone would achieve 92% accuracy.
This is a useful warning. A high accuracy value can look impressive while providing almost no clinical value.
Baseline: Predict no readmission for every patient.
Accuracy: 92%
Recall for readmission: 0%
The second number tells us why the first one is misleading. The model has failed to identify a single readmitted patient.
8. Choosing What to Measure
8.1 Accuracy Is Not Enough
Because the positive class is relatively uncommon, metrics that describe performance on the readmitted patients are especially important.
- Recall: Of the patients who were actually readmitted, how many did the model identify?
- Precision: Of the patients flagged as high risk, how many were actually readmitted?
- Specificity: How many patients who were not readmitted were correctly left unflagged?
- F1-score: How well do precision and recall balance at a chosen threshold?
- PR-AUC: How well does the model distinguish the relatively uncommon positive cases across thresholds?
8.2 The Cost of the Two Errors
Consider two patients:
- False Negative: A patient is predicted to be low risk but is readmitted shortly after discharge.
- False Positive: A patient is flagged as high risk but does not return to the hospital.
A false negative may mean that a patient who would have benefited from additional support was not prioritized. A false positive may consume limited case-management time. The appropriate balance therefore depends on what intervention the hospital is actually able to provide.
8.3 Evaluate the Decision, Not Just the Number
Suppose one model has recall of 0.72 and another has recall of 0.61. Is the first model automatically better? Not necessarily. The answer depends on precision, workload, threshold, intervention capacity, and the clinical consequences of the errors.
9. Prediction Is Not the Same as Clinical Action
Suppose a model assigns the following probabilities:
| Patient | Predicted probability of 30-day readmission |
|---|---|
| A | 0.08 |
| B | 0.27 |
| C | 0.54 |
| D | 0.81 |
The model does not automatically know what action should follow. The hospital may decide that patients above a certain threshold receive a post-discharge call, medication reconciliation, or expedited follow-up appointment.
If the hospital can support only 200 follow-up interventions per week, then the threshold may be chosen partly according to operational capacity. The optimal threshold is therefore a decision-design question, not simply a default setting in a software library.
10. Model Development
We can now train models using the algorithms learned throughout the course. We might begin with logistic regression, then compare it with a decision tree, random forest, and gradient-boosting model as those topics are covered later in the book.
10.1 Start Simple
Logistic regression is attractive as a baseline because its coefficients provide a relatively transparent starting point and its output can be interpreted as an estimated probability after appropriate modeling and calibration.
10.2 Compare Models Fairly
Every model should be evaluated using the same data partitions and a pre-defined evaluation procedure. Otherwise, the comparison itself can become unreliable.
10.3 Avoid the “Most Complex Model Wins” Trap
A more complex model may improve predictive performance, but it may also increase maintenance cost, reduce transparency, or behave differently under changing data conditions. The best model is not necessarily the model with the largest number of parameters or the most impressive validation score.
11. Error Analysis: Who Does the Model Miss?
Once a model has been selected, examine its errors rather than stopping at a single summary metric.
For example, compare recall and precision for patients with and without:
- diabetes;
- hypertension;
- ischemic heart disease;
- chronic kidney disease;
- multiple admissions during the previous year;
- emergency versus elective index admission.
A model can have excellent overall performance while behaving poorly for a smaller but clinically important group. The purpose of subgroup analysis is not to force every subgroup to have identical performance, but to reveal where the model behaves differently and investigate why.
11.1 An Example
Suppose overall recall is 0.74, but recall is only 0.51 among patients with chronic kidney disease. This does not tell us immediately what to do, but it tells us that the model deserves further investigation.
12. Interpreting a High-Risk Prediction
Imagine that the model flags a patient with diabetes, hypertension, chronic kidney disease, three prior admissions, and a long index stay as high risk. A clinician may reasonably ask:
“Why is this patient considered high risk?”
This question introduces interpretability. At a broad level, it is useful to understand which variables are generally influential in the model. At the individual level, an explanation can help clarify why a particular patient's predicted risk was high.
Later in the course, the dedicated Explainable AI material will introduce tools such as local explanations, surrogate models, feature-attribution approaches, LIME, and SHAP in much greater depth. Here, the important point is simply that a prediction used in a real decision context may need an explanation.
13. Deployment
The model is now ready to leave the notebook. What does deployment actually mean?
A practical workflow might look like this:
- Patient is admitted.
- Clinical information is recorded during the stay.
- At discharge, the system assembles the permitted features.
- The model estimates the probability of 30-day readmission.
- The risk estimate appears in the appropriate hospital workflow.
- The care team decides whether an intervention is warranted.
The model therefore becomes one component of a larger ML system. The model itself is not the entire system.
14. Monitoring
Good performance during model development does not guarantee equally good performance six months or two years later.
A hospital may change its admission policies, discharge practices, clinical guidelines, electronic-record system, referral patterns, or patient mix. The prevalence of disease may also change.
14.1 What Should We Monitor?
- Input distributions: Are the kinds of patients arriving today similar to those used during training?
- Prediction distributions: Is the proportion of patients being flagged as high risk changing unexpectedly?
- Outcome performance: Once sufficient follow-up time has passed, is the model still identifying readmissions accurately?
- Subgroup behavior: Has performance deteriorated for a particular patient group?
14.2 A Useful Thought Experiment
Imagine that a new hospital policy successfully reduces average length of stay. A feature that was historically very informative may behave differently after this operational change.
The model has not changed. The world around it has.
Machine learning does not end when a model reaches production. A deployed model is part of a changing socio-technical system and therefore requires monitoring.
15. End-to-End View of the Project
| Stage | What We Ask |
|---|---|
| Problem formulation | Can we identify patients at higher risk of unplanned 30-day readmission? |
| Target definition | What exactly counts as a readmission? |
| Prediction point | What information is available by the time of discharge? |
| Data understanding | What does each row represent and how were the variables recorded? |
| EDA | What patterns, missing values, outliers, and subgroup differences exist? |
| Preprocessing | How should missing and categorical data be handled without leakage? |
| Baseline | How much value does the model add beyond a trivial predictor? |
| Modeling | Which algorithms provide an appropriate trade-off between performance and practical use? |
| Evaluation | Which metrics reflect the real objective and consequences of errors? |
| Error analysis | Which patients does the model get wrong? |
| Interpretation | Can clinicians understand why a prediction was made? |
| Deployment | Where and how does the prediction enter the hospital workflow? |
| Monitoring | How will we know the model continues to work after deployment? |
16. What This Case Study Tells Us About Machine Learning
The healthcare setting differs from credit-risk assessment, but the underlying ML lifecycle is similar.
In both cases we needed to define the prediction task, understand the data, prevent leakage, establish a baseline, choose appropriate metrics, analyze errors, deploy the model within a larger workflow, and monitor performance over time.
What changes is the context.
In healthcare, the meaning of an error may involve a patient who needs additional support. The useful features may reflect medical history and clinical workflow. The prediction point is tightly constrained by what is known before discharge. The model may need to support rather than replace professional judgment.
Core lesson: Machine learning is not simply the process of selecting an algorithm and maximizing a metric. The data, prediction point, objective, errors, users, deployment environment, and changing real-world context all shape what a good ML solution looks like.
17. Interactive Examples
17.1 Which Variables Are Available at Discharge?
Evaluate whether each clinical variable is valid to use when predicting 30-day readmission at the exact moment of patient discharge.
Scenario A: Patient Age and Number of prior admissions.
Scenario B: Final length of stay for the current hospital visit.
Scenario C: Readmission within 30 days or an emergency visit 10 days post-discharge.
17.2 Is 92% Accuracy Good?
A 30-day readmission model achieves 92% overall accuracy on a hospital dataset where 8% of patients are readmitted.
17.3 What Would You Investigate First?
Suppose your model's overall recall is high, but its recall among patients with chronic kidney disease (CKD) is significantly lower. What should you do before changing algorithms?
- Sample Size: Is the CKD subgroup underrepresented in the training data?
- Data Quality & Missingness: Are key renal lab values (e.g., eGFR, creatinine) systematically missing or recorded differently?
- Feature Representation: Are disease-specific risk factors captured in the feature set?
- Error Profiling: Analyze false negatives within the CKD cohort to identify systematic error patterns.
18. Try It Yourself
The hospital wants to predict readmission immediately after a patient arrives in the emergency department. Which of the variables in the case study would no longer be valid?
Why might a random train/test split produce an over-optimistic estimate of performance when hospital practice changes over time?
Suppose the model identifies 300 patients as high risk, and 120 of them are actually readmitted. What is the precision?
$$\text{Precision} = \frac{\text{True Positives}}{\text{True Positives} + \text{False Positives}} = \frac{120}{300} = 0.40 \text{ (or } 40\% \text{)}$$
Interpretation: Out of all patients flagged as high risk by the model, 40% were actually readmitted.Suppose 160 of the 200 truly readmitted patients were identified. What is the recall?
$$\text{Recall} = \frac{\text{True Positives}}{\text{True Positives} + \text{False Negatives}} = \frac{160}{200} = 0.80 \text{ (or } 80\% \text{)}$$
Interpretation: The model successfully identified 80% of all patients who actually required readmission.A clinician says, “I will trust the model only if it can explain why this particular patient was flagged.” Which area of ML does this question lead toward?
19. Key Takeaways
- The first task is to define the prediction point and target precisely.
- Healthcare data must be interpreted in the context of clinical workflow.
- Predictive variables are not automatically valid variables; future information creates leakage.
- Class imbalance makes accuracy alone potentially misleading.
- The appropriate threshold depends on the action taken after the prediction.
- Error analysis should examine which patients the model gets wrong, not just the overall score.
- Interpretability can matter when predictions support professional decisions.
- Deployment places the model inside a larger hospital workflow.
- Monitoring is necessary because patients, practices, policies, and data change over time.
20. Common Pitfalls
- Using future information: A variable collected after discharge cannot be used for a prediction made at discharge.
- Reporting only accuracy: With an 8% positive rate, a trivial classifier can achieve 92% accuracy.
- Ignoring the prediction point: The same feature can be valid at discharge but invalid earlier in the admission.
- Ignoring missingness: Missing clinical information may reflect how care was delivered.
- Assuming the model is the decision-maker: The model estimates risk; people and workflows determine what action follows.
- Stopping at deployment: A model can deteriorate when the patient population or hospital workflow changes.
21. Looking Ahead
The case brings together several recurring themes in machine learning: classification, evaluation, class imbalance, error analysis, interpretability, deployment, and monitoring. These themes provide a foundation for the more detailed treatment of individual algorithms and techniques throughout the course.
Taken together, these stages show why individual ML techniques exist and where they fit within an end-to-end ML project.
A 2026 study from Aga Khan University and Aga Khan University Hospital in Karachi evaluated 35,496 adult internal-medicine admissions and found 8.0% 30-day readmission. In another Karachi tertiary-care study of acute severe hypertension, readmission was associated with conditions including raised blood pressure, acute myocardial infarction, and kidney failure. These findings provide context for the clinical setting described in this case study.
Selected References
- Abbas, M. et al. (2026). Assessing effectiveness of HOSPITAL score and LACE index for predicting 30-day readmissions in a tertiary care hospital in Pakistan: a retrospective cohort study. BMC Health Services Research.
- Assessing the impact of acute severe hypertension in the emergency department: A prospective cohort study in Karachi, Pakistan. PLOS Global Public Health.
- World Health Organization, Eastern Mediterranean Regional Office. Prevalence of risk factors for noncommunicable diseases in adults: key findings from the Pakistan STEPS survey.